Distributed file systems are generally employed for storage of large quantities of data and to reduce input/output (I/O) bottlenecks where there are many requests made for file access. In a distributed file system, the file data is spread across multiple data processing systems. File system control and management of file system meta-data is distributed in varying degrees in different systems.
A desirable characteristic of many distributed file systems is scalability. Scalability is a characteristic that refers to the ease with which a distributed file system can be expanded to accommodate increased data access needs or increased storage needs. For example, as additional users are granted access to the distributed file system, new storage servers may be introduced, and the requests of the additional users may be further spread across the old servers and new servers. The scalability of any distributed file system is limited or enhanced by the system design.
Caching is a feature that is commonly used to reduce data access times and to enhance the scalability of distributed file systems. However, caching requires additional management software to address data locking and data consistency issues. Thus, caching introduces additional complexity and overhead into a distributed file system.
Another approach that addresses scalability is the provision of dual paths for access to file data and access to meta-data. In this approach, the meta-data is managed on a server that is separate from the storage servers. However, this approach may create a bottleneck at the meta-data server and thereby restrict scalability.
A system and method that address the aforementioned problems, as well as other related problems, are therefore desirable.